1. Field of the Invention
The present invention relates to an imaging apparatus, a method of shooting an object and a computer program product, and more specifically, to an image apparatus, a method of shooting an object and a computer program product, which are used suitably, for instance, in digital cameras and the like for obtaining photographs having good image quality even when shooting with a long exposure time.
2. Description of the Related Art
FIG. 12 is a view showing a general idea of the a first conventional technique disclosed in Japanese Patent Application 2001-326850A.
In FIG. 12, T1 denotes an exposure time for shooting an object under low light conditions where it is prohibited to use a strobe light.
The exposure time T1 is an exposure time that is longer than an appropriate time for shooting under good light conditions.
In the first conventional technique, an image signal is read out from the two-dimensional image sensor several times during the exposure time T1.
For instance, the exposure time T1 is evenly divided into four exposure times, T1(1) to T1(4), as shown in FIG. 12, and images are read out from the image sensor respectively in four exposure times T1(1) to T1(4). Note that the exposure time T1 is not always divided evenly but may be simply divided into plural exposure times.Finally, these four images 1 to 4 are combined together to produce one image signal 5. In FIG. 12, sizes of four images 1 to 4 are different from the combined image 5, but the image 5 is drawn larger than other images 1 to 4 for the sake of explanation. In practice, the four images 1 to 4 have the same size as the combine image 5.
In a system of combining plural images into one image, the exposure time (T1(1) to T1(4)) for each of the images 1 to 4 is obtained by dividing the exposure time T1 by 4, where the exposure time T1 is an exposure time for shooting an object under low light conditions where it is prohibited to use the strobe light. Therefore, when the exposure time (T1(1) to T1(4)) for the images 1 to 4 is obtained by dividing the exposure time T1 evenly, brightness of the images 1 to 4 would have been one fourth of target brightness.
When the exposure time T1 is divided evenly for each of the images 1 to 4, brightness of the image 5 obtained by combining these four images 1 to 4 corresponds to approximately four times of brightness of each of the images 1 to 4. As a result, an image (combined image 5) can be obtained, of a predetermined brightness corresponding to the exposure time for shooting an object under low light conditions where it is prohibited to use the strobe light.
A dark current component (a level of which is assumed to be “P” for convenience) contained in the combined image 5 corresponds approximately to the total (≈P(1)+P(2)+P(3)+P(4)) of dark current components (levels of which are assumed to be P(1), P(2), P(3) and P(4) respectively for convenience) contained in each of the images 1 to 4.
As described above, the dark current is a sort of sensor noises that increase exponentially with exposure time.
Assuming that a dark current component having a level of P′ is contained in one image photographed with the exposure time T1, “P′>P” will be true. Therefore, the dark current component “P” contained in the combined image 5 can be decreased, whereby the image photographed with a long exposure time is improved in image quality.
The first conventional technique mentioned above has advantages in improving image quality of an image photographed with a long exposure time. But on the contrary the first conventional technique has disadvantages that cannot solve problems caused due to camera shake, which occurs frequently.
Japanese Patent Application 2005-182089 A describes a technique (hereinafter, a “second conventional technique”) which takes measures for camera shake.
FIGS. 13A and 13B are views illustrating an idea of the second conventional technique.
In FIG. 13A, four images 6 to 9 correspond respectively to the images 1 to 4 obtained in the first conventional technique as shown in FIG. 12. In FIG. 13B, an image 10 corresponds to the combined image 5 in the first conventional technique as shown in FIG. 12.The second conventional technique is different from the first conventional technique in that objects (persons) of four images 6 to 9 are shifted in position a little from each other in the images.More specifically, when compared with the position of the person 11 in the first image 6, the person 12 of the second image 7 is shifted in position a little to the right by an amount of F1 as shown in the image 7 of FIG. 13A.Further, when compared with the position of the person 12 in the second image 7, the person 13 of the third image 8 is shifted in position a little to the right by an amount of F2 as shown in the image 8 of FIG. 13A.Similarly, when compared with the position of the person 13 in the third image 8, the person 14 of the fourth image 9 is shifted in position a little to the right by an amount of F3 as shown in the image 9 of FIG. 13A.The persons 11 to 14 appearing in the four images 6 to 9 are the same person, and therefore these four images indicate that the camera are shaken in the leftward while these images are photographed.The second conventional technique has been described with respect to the camera which has been shaken in the horizontal direction. The second technique can be applied to the camera which has been shaken not only in the horizontal direction but also in the vertical direction or in the other direction.
In the second conventional technique, feature points of edges and feature points of hue are extracted from each of the four images 6 to 9, and the extracted feature points are checked to specify common feature points.
Plural feature points common to these images 6 to 9 are calculated to determine their relative positions, and relative positions of the images 6 to 9 are determined based on the determined relative positions of the feature points. Then, positions of the four images 6 to 9 are corrected based on the determined relative positions of the four images 6 to 9 to combine the four images together, thereby generating one combined image 10.In short, movement or displacement between two images which are adjacent in terms of time is detected and then the detected movement or displacement is corrected to generate the combined image 10.
As described above, the second conventional technique solves problems caused due to camera shake by correcting the movement or displacement between images adjacent in terms of time.
And further, dark current components contained in the combined image 10 is reduces, whereby an image photographed with a long exposure time is improved in quality.
However, the second conventional technique has another disadvantages that it takes a long time to compensate for limb darkening of an optical system, as described below.
FIG. 14 is a view illustrating limb darkening phenomenon of the optical system. In FIG. 14, the bold circle outermost among a number of concentric circles denotes an image circle 15 of the optical system.
A landscape rectangle within the image circle 15 denotes an imaging area 16 of the two-dimensional image sensor such as CCD. An image 18 shown below the image circle 15 in FIG. 14 represents an image output from the two-dimensional image sensor, which includes an image of a person-like object in the vicinity of the center of the imaging area 16.
In general, an optical system of a camera, brightness at a point on an image is determined depending on characteristics of the optical lens and has “limb darkening characteristics”. That is, the brightness at a point on an image is given by cos4θ, where θ denotes an angle between a line connecting the point on the image with an optical lens and an optical axis of the optical lens. In short, a point on an image produced through the optical lens darkens gradually as the point moves from the center of the image to the edge.
In FIG. 14, a number of concentric circles arranged within the image circle 15 schematically indicate the limb-darkening characteristics. A closer space between the circles expresses that darkening rate is larger.
In most today's cameras, various devices such as special combination of optical lenses are made to solve or remove problems caused by the limb-darkening characteristics of the optical lens.
In actual situation, the above devices need substantial production costs, and therefore it is common that these devices are employed only in some single-lens reflex cameras.
In widely used cameras, the limb darkening is accepted to some extent, but most cameras employ a sort of image processing for compensating for the limb darkening.
Japanese Patent Application 2002-296685 A describes a technique (hereinafter, a “third conventional technique”) that increases brightness at a point on the image 18 shown in FIG. 14 as the point moves to edge of the image, thereby solving the problem of limb darkening without increasing costs.
The limb darkening effect appears in the original image 18 as illustrated by spaces between concentric circles shown within the image 18. The third conventional technique uses such proper compensation data as cancels the limb darkening effect in the image 18 to make brightness distribution even and/or flat over the image 18.
However, when the third conventional technique (a method of processing an image to compensate for limb darkening in the image) is applied to the second conventional technique without any modification made to said third conventional technique, it is hard to make brightness distribution flat or even over the finally obtained image 10 which is subjected to limb darkening effect.
FIG. 15 is a view illustrating disadvantages caused by the conventional techniques.
In FIG. 15, a process is shown, for simplicity, in which two images (first image 19 and second image 20) which have been photographed continuously are combined together to generate one combined image 21.
A person 22 appears approximately at the center of the first image 19, and in the second image 20 the same person 23 appears at a position shifted slightly in a lower rightward direction from the position of the person 22 in the first image 19. That is, camera shake makes the position of the person 23 in the second image 20 shift from the position of the person 22 in the first image 19.As a result of alignment of the person 22 in the first image 19 and the person 23 in the second image 20 in the second conventional technique, the combined image 21 is obtained with a person 24 appearing approximately at the center thereof.
Blacked out portions of the first image 19 and the second image 20 in FIG. 15 schematically represent a portion 25 of low brightness in the first image 19 and a portion 26 of low brightness in the second image 20, respectively. These portions of low brightness are yielded due to limb-darkening characteristics of the optical system of the camera.
In reality, brightness varies linearly in the portion of low brightness yielded due to the limb-darkening characteristics, and therefore the portion of low brightness is by no means simply blacked out.
But for simplicity in explanation, it is assumed that brightness varies in two steps as illustrated by white portions (portions of high brightness) in the vicinity of the persons 22, 23 and blacked out portions (portions of low brightness) 25, 26.
In the case where the first image 19 including the low-brightness portion 25 and the second image 20 including the low-brightness portion 26 are aligned to produce the combined image 21, low-brightness portions due to the limb-darkening characteristics are made complex in shape in the combined image 21 as shown in FIG. 15.
That is, since the second image 20 is shifted by a predetermined distance in an upper leftward direction to be aligned with the first image 19 whereby the combined image 21 is produced, portions of low brightness included in the combined image, due to the limb-darkening characteristics consist of the low-brightness portion 25 of the first image 19, low-brightness portion 26 of the second image 20, and a superimposed portion (medium-brightness portion) 27 of the two low-brightness portions 25 and 26.
As described above, the combined image 21 includes not only low-brightness portions 25, 26 but also the medium-brightness portion 27, which is brighter than the low-brightness portions 25, 26. Therefore, when simple compensation is made to the combined image based on the limb-darkening characteristics of the optical system to as to compensate for low brightness in the combined image, the medium-brightness portion 27 would have been compensated for excessively.
The compensation process described above is not practicable to make brightness distribution (low-brightness portions 25, 26 and medium-brightness portion 27) due to the limb darkening even or flat in the finally obtained image.
Therefore, when the second conventional technique is used to compensate for limb darkening in the image, limb darkening in the first and second images 19, 20 shown in FIG. 15 must be compensated for separately, before these images 19 and 20 are combined together.
As a result, a problem is yielded that the more images have to be compensated, the longer time is required to compensate for limb darkening in the images.